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ABSTRACT 



This paper discusses the results of a study that 
investigated the effectiveness of the Fort Bragg demonstration project, a 
program which provided a comprehensive approach to the delivery of mental 
health and substance abuse services to a population of approximately 48,000 
military-related children residing within the Fort Bragg catchment area. Over 
a 6 -month period, more than 200 individuals were interviewed, including 
administrators, clinicians, families, policymakers, and other stakeholders. 
Through the study, there emerged greater clarity as to what the Fort Bragg 
demonstration project was and was not. Fort Bragg was not managed care; it 
sought specifically to increase access and utilization without particular 
regard to cost. The demonstration also was not a system of care or a 
continuum for high end users. It was a demonstration of a community-based 
continuum of mental health and substance abuse services with a single point 
of access. It expanded the array of services, eliminated co-pays and 
deductibles, reduced use of inpatient and residential treatment, increased 
access and utilization, and was held in generally high regard by families and 
the larger community. Critical lessons learned are also discussed. (CR) 
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Introduction 



The Fort Bragg demonstration provided a comprehensive 
approach to the delivery of mental health and substance 
abuse services to a population of approximately 48,000 
(FY95) military-related children under 18 years of age who 
resided within the Fort Bragg catchment area. A major 
purpose of the project was to study the implications of 
expanding the health care benefit package provided to 
military families through the Civilian Health and Medical 
Program of the Uniformed Services (CHAMPUS) by 
providing a full continuum of mental health and substance 
abuse services (Bickman et al., 1995). 



As an experiment in systems change, the Fort Bragg 
demonstration yields a wealth of valuable "lessons learned" 
for policymakers, practitioners, researchers, and consumers. 
These lessons have implications both for those involved in 
children's systems reform efforts, as well as those 
undertaking managed care initiatives a ffe cting children and 
their families. 
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Method 



Over the last six months of the demonstration, over two 
hundred individuals were interviewed, including 
administrators, clinicians, families, policymakers, and other 
stakeholders, who had been involved in the Fort Bragg 
project since its inception. Interviews were conducted by a 
team of independent policymakers, researchers, family 
members, and clinicians, all of whom have been involved in 
national and state systems reform issues over the last 
decade. The team also reviewed the extensive written 
documentation available from the demonstration, the formal 
j evaluation conducted by Vanderbilt University, and client 
1 characteristic and service data over the life of the project. 

I What Fort Bragg Was and Was Not 

I Through the study, there emerged greater clarity as to what 
the Fort Bragg demonstration was and was not. 

■ Fort Bragg was not managed care. The demonstration 

I sought specifically to increase access and utilization without 
j particular regard to cost, in contrast to managed care, which 
j is concerned fundamentally with controlling cost by 
j managing access and utilization. There was no risk attached 
1 to management and service contracts (which were 
cost-reimbursable); many incentives existed, on both the 
supply and demand side, to provide and use services. 

I The demonstration also was not a system of care; that is, it 
I was not an integrated continuum of mental health and 
I related services and supports in which there is shared 
: "ownership" of the system across multiple child-serving 

■ systems (i.e., education, child welfare, juvenile justice and 
; mental health). 

■ Fort Bragg was not a continuum for "high end users" (i!e., 
i children with serious disorders, only). Nearly 60% of the 

; evaluation samp^le received no service more intensive than 
I outpatient. 

Fort Bragg was a demonstration of a community-based 
continuum of mental health and substance abuse services, 
with a single point of access, for a total child/adolescent 
eligible population (i.e., for both acute and extended care 
populations and for children with both mild and serious 
disorders). It changed an existing, poorly regarded system, 
as it intended; it expanded the array of services, eliminated 
co-pays and deductibles, reduced use of inpatient and 
residential treatment, increased access and utilization, and 
was held in generally high regard by families and the larger 
community. With this success, however, there emerged 
countervailing pressures and issues (e.g., cost pressures, 
overutilization of multiple services, unrealistic expectations 
on the part of some families and clinicians, and apparent 
cost shifting to and from the demonstration). To its credit, 

I the demonstration made every effort to be a self-correcting 
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the demonstration made every effort to be a self-correcting 
experiment. 

Critical Lessons Learned 

j 

Service Utilization 

At least two important lessons emerge from looking at the 
greatly expanded service utilization created by the 
demonstration. The first is the "if you build it, they will 
come" lesson. If mental health services are offered to 
families in ways that are accessible, stigma-reducing, and 
individualized, they will be used&endash;and probably 
more heavily than a developing system can accommodate 
without creating problems in other areas, such as cost 
problems, inadequate provider training, inadequate 
I monitoring, and quality assurance. A response to these 
i problems is not necessarily to restrict access, which remains ' 
I a worthwhile policy goal, but rather to design the 
j demonstration and manage services during implementation 
; in ways that promote more deliberative start-up and efficient : 
i treatment. 

I A second lesson from looking at service utilization in Fort 
Bragg is the importance of analyzing as accurately as 
possible the population eligible for services prior to 
implementation (i.e., who and how many can be expected to 
i use services). This requires striking some balance between 
i using prior utilization and prevalence data. The Fort Bragg 
' demonstration was hampered by the inadequacy of the 
J Army's data regarding how many children were using 
services and the types of services being used, as well as the 
insistence to base expected use on prior utilization only. 

There is a particular lesson here for managed care initiatives 
which have a tendency to rely on prior utilization to develop ■ 
: capitation rates. The Fort Bragg project suggests strongly 
i that use of prior utilization alone will be an inaccurate 
I predictor if it is applied to a more attractive, accessible, 
j community-based continuum of behavioral health services. 

j Fort Bragg points to the need for individualized system 
I planning for different subclusters of children in a total 
I eligible population. For example, while "high end" users 
1 may need intensive case management and treatment 

planning, this is clearly not the case for children using only 
outpatient services. Children involved in other public 
systems, particularly child welfare, will require more time 
on the part of clinicians to manage the boundaries between 
the systems and deal with the requirements of the other 
system. Children with ADHD may require less intensive 
involvement with mental health but greater collaboration 
with the pediatric services. Fort Bragg suggests an 
I important lesson about the need to desegregate the eligible j 
I population into types of users and institute approaches that 
i m^e sense for different subpopulations&endash;the "not 
I every child needs the same system" lesson. 

i Cost 
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Perhaps the most important cost-related lesson from Fort 
Bragg is that of linking cost and quality issues in design and 
implementation&endash;a lesson which Fort Bragg itself 
I learned, somewhat painfully, and took steps to change. It is 
I not possible to maintain quality of care without paying 
attention to cost; clinical viability alone is not sufficient. 
Inevitably, costs will run high if not attended to from the 
I outset, leading to enormous pressures for cost containment 
that then can jeopardize quality of care. Cost containment 
pressures, in turn, will aggravate tendencies of cost-shifting 
to other child-serving systems. 

A related cost lesson is the importance of thinking through 
I in advance the effect on supply and demand of the 
I incentives that are built into the system for providers and 
I consumers to provide and use services, respectively. To its 
I credit. Fort Bragg took a number of steps by the latter half 
I of the demonstration (post the evaluation phase) to modulate 
I some of these incentives and control costs without 
I necessarily jeopardizing quality of care. It also instituted 
; greater efficiencies in service delivery through the 
i following: (a) streamlining its intake process; (b) 

1 emphasizing shorter-term, problem solving therapies; (c) 

: adhering more closely to level of care criteria; (d) instituting 
‘ guidelines for use of psychiatrists; (e) streamlining its case 
i management process; and (f) streamlining the treatment 
: planning process. It created more structured utilization 
review, with a clearer focus on length of stay, and imposed a 
; more integrated focus on cost, quality, and service issues 
i through its quality improvement process with feedback 
j loops to service providers about both cost and quality. In 
I this process, the demonstration also had to begin to clarify 
■ its own values about cost and quality issues&endash;again, 

' an important lesson. When cost containment values were 
; introduced into the service system late in the demonstration, 

: there was indeed a dramatic reduction in the cost of 
providing treatment. 

I A strong, argument can be made from the Fort Bragg 
experience that research and development costs and costs 
associated with requirements imposed by demonstration 
sponsors should be budgeted and tracked separately from 
the costs associated with service delivery. Fort Bragg also 
suggests (as do many other demonstrations in the children's 
arena) that the time normally accorded to systems change 
efforts (typically, three to five years) is simply too short to 
• preclude inefficiencies in start-up, development, and 
j modification of new approaches (Friedman, 1 996). 

j 

j A final cost lesson from Fort Bragg relates to the possibility 
j that greater efficiencies may very well have been achieved, 

I at least with respect to Fort Bragg children involved in 
I multiple systems, had the demonstration actually 
I implemented a system of care with shared ownership across 
I the major child-serving systems, instead of a continuum of 
I mental health services "owned" by the Army. Certain 
’ inefficiencies occurred because separate systems prevailed. 
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inefficiencies occurred because separate systems prevailed. 
Service Delivery 

A great many lessons can be drawn from the demonstration 
about effective service delivery. 

i 

Regarding intake and assessment; 

• Intake staff need access to an array of crisis options; 

• intermediate-level services, particularly in-home 
services, need to be available from the outset to 
prevent inappropriate hospitalization and residential 
care; 

• an intake process that focuses on presenting problems, I 
using a problem list, rather than on "nailing a 
diagnosis" is needed to promote efficiency; 

• small co-pays are needed to prevent an initial 
recurring problem of missed appointments; and 

• intake staff need to be trained well in the treatment 
philosophy and goals of the demonstration, how the 
service system is organized, and the resources 
available to them. 

Regarding the treatment team process: 

• It is critical to make the treatment team responsible 
for managing/monitoring length of stay and 
appropriateness of care to better link treatment and 
cost concerns; 

• a multidisciplinaty treatment team does not have to be i 
held for every child entering the system but should be ' 
reserved for children with serious or complex 
disorders; 

• the more people who are involved in the treatment 
team process, the more time-consuming and 
expensive it will be (i.e, multiple players on the 
treatment team may aggravate the problem of overuse : 
of multiple services, since each player has a tendency 

to become an advocate for the service he or_she is 
representing, which also drives up costs); 

• an effective treatment team process is one governed 

by tight clinical leadership and a better understanding I 
of how the various levels of care relate to one another; ; 

• much more needs to be learned about what service 
components work best for which types of children 
over what period of time; and 

• it is important to be creative in treatment planning and i 
not allow a child to stay in an inappropriate level of 
care simply because the needed service is unavailable. 

Quality Improvement 

A number of lessons can be drawn about quality 
improvement (QI). The QI program should be developed 
with the involvement of staff and providers, as well as 
families, and incorporate peer review to create ownership 
and "buy-in." QI should provide information to clinicians 
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and "buy-in." QI should provide information to clinicians 
concurrently with service provision, not retrospectively, so 
that feedback in areas such as length of stay and family 
satisfaction gets to providers in a timely fashion, not after 
I the fact. QI should track more than cost and utilization; it 
needs also to track systems outcomes across the continuum, 
client outcomes, and family and youth satisfaction. QI and 
utilization review functions need to be integrated. QI data 
needs to be tied to credentialling of providers. It is 
unrealistic to expect a full-blown QI program to be in place 
before clinical services are fully developed. Implementation 
of an effective QI program is a developmental process that 
requires time. 

Staff and Provider Network 

One of the lessons of Fort Bragg is the importance of 
orienting the staff and provider network to the values and 
i goals of the demonstration and of evaluating staff and 
I providers against the demonstration's desired outcomes, 
j Because differing values are inevitable among staff and 
I providers, it is essential to articulate clearly the 
I demonstration's values as a unifying element and to create 
an in-house process to air and resolve differing values. 

I Without such a process, staff become polarized around 
different philosophies, instead of working synergistically. 

Family Involvement 

Perhaps the most salient lesson about family involvement is 
the danger in assuming that everyone means the same thing 
I by "family involvement," the importance of articulating 
; clearly what the demonstration means by it and the 
I importance of training staff in the demonstration's 
i philosophy and service approach to families. There needs to 
! be clinical leadership to help families make good decisions, 
i Lack of strong clinical leadership, particularly at a systems 
: level, affects cost. In the absence of such leadership, there is 
; a tendency for staff and providers to overload families with 
I services and for families to become dependent, aggravating 
. lengths of stay and therefore cost. 

Environmental Context 

Systems change experiments in the children's arena operate 
i in complex political, social, and interagency environments. 

I Particularly in the children's arena, changes in one system 
I will affect all other child-serving systems. 

I 

I Fort Bragg reinforces the usefulness of environmental and 
I ethnographic analysis for anticipating how services will be 
i used and for informing deliberations about optimal 
i treatment approaches for the target population. For example, 
' it might be anticipated that ready access would be a major 
I factor in the willingness of Army families to use services 
I because of the Army mentality of "do it now." Similarly, the 
i location of services off-base might also have been 
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anticipated to increase demand for services because of the 
preference of military families to use non-base providers, 
particularly for mental health services where stigma is an 
issue. The transiency of the military family, particularly 
those at this rapid deployment base, might have argued in 
the design phase for shorter-term, problem-solving 
treatment approaches and shorter lengths of stay. The rapid 
i deployment nature of the base, which has the efect of 
depleting medical staff from the base, might also have been 
anticipated to result in some patient-shifting from the Army 
medical center on-base to the demonstration, as indeed 
occurred. 

Evaluation 

I Just as it is essential for demonstration implementors to be 
I clear about their purpose, it is critical that evaluators are 
I careful to define clearly what it is they are evaluating. It also 
I is critical that appropriate caveats be included regarding 
i comparison sites that are used. Typically, there are political, 

I operational, and other constraints that influence the choice 
I of comparison sites, which necessarily affects 
I comparability. Those involved in the demonstration believe 
I that the most, perhaps only, comparable site to Fort Bragg is 
1 Fort Hood, which, for a number of reasons, could not be 
I used (L. Behar, personal communication. Summer, 1995). 

I Both bases, unlike the comparison sites that were used, are 
j involved in rapid deployment, house specialized forces, 

: have large pediatric populations, have high birth rates, and 
i have a large concentration of low income military 
I personnel. It is possible that these factors create more 
I difficult-to-treat mental health problems. If this is the case, 
i then the fact that Fort Bragg had outcomes as good as, and 
j in some cases, better than the comparison sites that were 
! used, is encouraging. Similarly, a caveat might have been 
I noted regarding the potential influence on outcomes of 
j co-pays and deductibles, with which families at the 
comparison sites, but not Fort Bragg families, had to 
contend. Families at the comparison sites may have been 
more motivated, due to the presence of co-pays and 
deductibles, to use services and to use them more 
efficiently. This motivation, in turn, may have helped to 
produce better treatment outcomes. 

In assessing adequacy of implementation, evaluators need to 
be clear about the stage of implementation that is being 
evaluated, as systems change experiments are highly 
developmental in nature. They also need to define carefully 
what aspects of implementation are being assessed and the 
limitations of those aspects to creating a holistic picture of 
implementation. The Fort Bragg evaluation, for example, 
j evaluated case management and intermediate services as 
I indicators of adequacy of implementation. The evaluation 
I did not assess the appropriateness of case management for 
I the various populations of children receiving that service, 

I which proved to be an issue. The evaluators assessed 
j intermediate services in one clump, rather than separately. 
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making it impossible lo determine whether certain 
components were more or less well developed than others, 
which might affect results. Also, at the time of the 
implementation evaluation, the demonstration was at a 
relatively early stage of development, besieged by a heavy 
demand for services, an influx of new providers, and 
without well developed internal management and control, 
quality assurance and utilization review mechanisms, all of 
which affected results. 

Evaluators need to be very careful in their interpretation of 
cost findings. In Fort Bragg, one might have concluded that 
the high costs associated with the demonstration were 
exactly what might have been anticipated given its design. 
This is a quite different interpretation than saying that costs 
were higher than expected. This goes back to the earlier 
point about the importance of both the demonstration and 
I the evaluation being clear about purpose and goals. 

; Summary 

I Demonstrations are expected to undertake many new 
I endeavors very quickly. Fort Bragg, for example, had to 
I simultaneously educate the community, hire new staff, 

I contract with outside providers, start new programs, build 
I internal management and monitoring mechanisms, respond 
I to sponsor demands for information and adjustments, orient 
and train new staff and providers, find facilities, etc. 
Multi-faceted systems, changing experiments in the 
children's arena, need time and a deliberately phased 
timetable, with benchmarks along the way, to be 
implemented (and evaluated) effectively. 

j 

I References 

j 

I Bickman, L., Bryant, D. M., & Summerfelt, W. T. (1993). 

I Final report of the quality study of the Fort Bragg 
Evaluation Project. Vanderbilt Institute for Public Policy 
Studies, Center for Mental Health Policy, Nashville, TN. 

Bickman, L., Guthrie, P. R., Foster, E. M., Lambert, E. W., 
Summerfelt, W. T., Breda, C. S., Heflinger, C. A. (1994). 
Final report of the outcome and cost/utilization studies of 
the Fort Bragg evaluation project. Vanderbilt Institute for 
Public Policy Studies, Center for Mental Health Policy, 
Nashville, TN. 

Bickman, L., Guthrie, P. R., Foster, E. M., Lambert, E. W., 
Summerfelt, W. T., Breda, C. S., & Heflinger, C. A. (1995). 
Evaluating Managed Mental Health Services: The Fort 
Bragg Experiment. New York: Plenum Press. 

Bickman, L., & Heflinger, C. A. (1993). Final report of the 
implementation study of the Fort Bragg Evaluation Project. 
Vanderbilt Institute for Public Policy Studies, Center for 
Mental Health Policy, Nashville, TN. 



9 



Friedman, R. M. (1996). Services and service delivery 
systems for children with serious emotional disorders: 

Issues in assessing effectiveness. In C. T. Nixon and D. A. 
Northrop (Eds.). Evaluating mental health services: How do 
I programs for children "work" in the real world?" 



Authors 

Sheila A. Fires, M.P.A. 

Partner 

Human Service Collaborative 
2262 Hall Place, NW, Suite 204 
Washington, DC 20007 
Voice: 202/333-1892 
Fax: 202/333-8217 
sapires@aol.com 

Lenore Behar, Ph.D. 

Head, Child and Family Services Branch 
Developmental Disabilities and Substance Abuse 
Services 

325 N. Salisbiuy Street, Room 523 
Raleigh, NC 27603 
Voice: 919/733-0598 
Fax: 919/715-5722 

Robert M. Friedman, Ph.D. 

Chair 

Department of Child and Family Studies 
Florida Mental Health Institute 
University of South Florida 
13301 Bruce B. Downs Boulevard 
Tampa, FL 33612-3899 
Voice: 813/974-4640 
Fax: 813/974-4406 

Ira Lourie, M.D. 

Partner 

Human Service Collaborative 
2262 Hall Place, NW, Suite 204 
Washington, DC 20007 
Voice: 202/333-5998 
Fax: 202/333-8217 

Beverly Ferreiro, Ph.D. 

Department of Psychiatry 
Box 3454 

Duke University Medical Center 
Durham, NC 27710 
Voice: 919/687-4676 
Fax: 919/687-4737 

David Langmeyer, Ph.D. 

909 Vestivia Woods Lane 
Raleigh, NC 27615 
Voice: 919/847-0098 
Fax: 919/848-3288 




10 





Fax: 919/848-3288 




Ann May 
2908 Debra Drive 




Raleigh, NC 27607 
Voice: 919/787-8235 




Fax: 919/783-6038 


1 


Katherine J. Lazear, M.A. 

Department of Child and Family Studies 
Florida Mental Health Institute 
University of South Florida 
13301 Bruce B. Downs Boulevard 
Tampa, FL 33612-3899 
Voice: 813/974-6135 
Fax: 813/974-4406 


1 


lazear(^al . fmhi . usf. edu 



Proceedings, A System of Care for Children's Mental Health: Expanding the Research Base (February 
26 - 28, 1996). Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health 
Institute, Research & Training Center for Children's Mental Health. 




11 



3o(r)gSL 




U.5. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




NOTTCE 

REPROnUCTTON BASIS 




This document is covered by a signed “Reproduction Release 
(Blanket) form (on file within the ERIC system), encompassing all 
or classes of documents from its source organization and, therefore, 
does not require a “Specific Document” Release form. 




This document is Federally-funded, or carries its own permission to 
reproduce, or is otherwise in the public domain and, therefore, may 
be reproduced by ERIC without a signed Reproduction Release form 
(either “Specific Document” or “Blanket”). 




EFF-089 (9/97) 





